home
***
CD-ROM
|
disk
|
FTP
|
other
***
search
/
SunSoft Catalyst CDWARE 1996 May to August
/
Catalyst CDWARE 1996 May to August.iso
/
.products
/
Apogee
/
_text
/
BenchResults.txt
next >
Wrap
Text File
|
1996-02-16
|
22KB
|
538 lines
This file contains the detailed results of the benchmark comparisons
between the Apogee 2.3 and compilers from various competitors,
including:
SunPro 3.0 (C and FORTRAN)
Fujitsu F90 1.0 (C and FORTRAN)
GNU C 2.3.3 (C only)
CenterLine 1.0.4 (C only)
The benchmarks were run in May 1994 on a SPARCstation 10 Model 30,
with the following characteristics:
SPARCstation 10 Model 30
36 MHz TMS390Z50 CPU with integrated FPU (single CPU)
Primary Cache: 20 KB Instruction, 16 KB Data (on chip)
Secondary Cache: None
Memory: 32 MB
Operating System: SunOS 5.2 (Solaris 2.2)
File System: UFS
Disk Subsystem: Fujitsu M2654SA (2GB) SCSI
System State: Multi-user, single user logon
Machine and OS choice were based on the following considerations:
The SunPro 3.0 and Fujitsu compiler suites are only available on
Solaris 2, so we chose that OS. The most recent Solaris 2 release
we have in house which is a customer-available, generally functional
release is version 2.2. We have a SPARCstation 10/30 (which was used)
and a SPARCstation 10/512 capable of running that release of the OS.
We chose not to use the (multiprocessor) 512 for two reasons: the
presence of an external cache (1MB per processor in this case) causes
very unreproducable performance runs; and not all compilers are
capable of making effective use of multi-processors (though we are),
and we didn't want to stack the deck in our favor. Interested
customers are encouraged to contact us for comparison results on
multi-processor machines.
We have made every effort to achieve the best performance with
everyone's compiler, including our own. We requested optimal flags
and official results for the SPEC benchmarks from SunPro and Fujitsu,
but neither gave them to us. We found makefile wrappers for
Metaware's compiler (CenterLine's compiler is based on Metaware
release 2.5), but were able to get much better results than those
wrappers gave. We are using the latest released compilers from each
vendor available to us on the date of the tests (there probably is a
newer version of GNU's compiler available, but we haven't set it up).
We welcome any suggestions for improvements in flags for any of the
compilers. We especially welcome suggestions from the respective
suppliers!
We started with previous SPEC published benchmark results. We tried
adding various flags documented in the various manuals. We ran
numerous experiments trying various combinations of flags. The best
result has been used in each case.
It is our belief that these represent the best case, "apples-to-apples"
comparisons achievable with our resources.
Once the best combination of flags was determined, each benchmark was
run three times, with the best result used.
For SPEC, "wall-clock time" is reported, as usual.
For Lawrence Livermore Loops we have reported the Harmonic Mean, as usual.
All vendors had C compilers, so we were able to obtain comparable
CINT92 numbers for all vendors. Apogee, SunPro and Fujitsu also offer
FORTRAN compilers, which we used (along with the C compilers) for the
CFP92 and LLNL (Livermore) FORTRAN Kernels.
To generate the graph, the SunPro compilers were used as a baseline.
Everyone's performance was divided by the SunPro performance, and then
converted to a percentage of the SunPro numbers. Hence, SunPro always
got exactly 100% of their own performance (by definition). As it
turned out, all other compilers got worse performance on the aggregate
CINT92, CFP92 and Livermore numbers, and Apogee beat everyone on each
of the aggregate numbers.
The lists below contain performance data and flags used for each benchmark.
=============================================================================
Performance Summary
===================
For all the summary numbers here, higher numbers are better.
The CINT92 benchmark is designed to measure CPU performance on various
"system" and/or "integer" computation. There are 6 benchmarks, all
written in the C language. The results can be combined into an
aggregate number labeled "SPECint92":
Apogee: 50.4 SunPro: 46.7 Advantage: Apogee 8%
Fujitsu: 41.9 Advantage: Apogee 20%
GNU: 41.4 Advantage: Apogee 22%
clcc: 40.0 Advantage: Apogee 26%
The CFP92 benchmark is designed to measure CPU performance on various
floating-point intensive computation. There are 14 benchmarks, 12 of
which are written in various dialects of FORTRAN-77 and 2 of which are
written in the C language. The results can be combined into an
aggregate number labeled "SPECfp92":
Apogee: 55.3 SunPro: 49.3 Advantage: Apogee 12%
Fujitsu: 42.3 Advantage: Apogee 31%
The Lawrence Livermore National Laboratories FORTRAN Kernels is
designed to measure CPU performance on 24 loops, each of which is
designed to model the computationally intensive kernel in a different
floating-point intensive application. The benchmark is contained in
one program which is written in a dialect of FORTRAN-77. The results
can be combined using a Harmonic Mean or a Geometric Mean. The
Harmonic Mean is reported here (the Geometric mean is shown at the end
of this file):
Apogee: 9.34 SunPro: 7.35 Advantage: Apogee 27%
Fujitsu: 6.93 Advantage: Apogee 35%
The lists below contain performance data and flags used for each benchmark.
=============================================================================
===============================================
| SPEC 1992 Integer CPU Benchmark (SPECint92) |
===============================================
For all times shown, lower numbers are better. The time is the wall-clock
runtime of the benchmark (in seconds). Short labels are used as
follows:
"Apogee" represents the Apogee AC2.3 compiler suite
"SunPro" represents the SunPro SC3.0 compiler suite
"Fujitsu" represents the Fujitsu F90 version 1.0 compiler suite
"GNU" represents the GNU C version 2.3.3 compiler
"clcc" represents the CenterLine C version 1.0.4 compiler,
which appears to be based on the Metaware v2.5 compiler.
008.espresso:
Times: Apogee: 52.6 SunPro: 53.2 Advantage: Apogee 1%
Fujitsu: 57.6 Advantage: Apogee 9%
GNU: 52.8 Advantage: Apogee 0%
clcc: 54.5 Advantage: Apogee 4%
Apogee Flags:
OPT=-O4 -K -w -Xintedge,T=ss10/30
EXTRA_LDFLAGS=-dn
EXTRA_LIBS=-L/usr/ucblib -lbsdmalloc -lucb
used src.alt/espresso.h.apx
SunPro Flags:
OPT=-fast -O4 -cg92 -Qoption cg -QrmW -Qoption cg -QrmX -Qoption cg -QrmR
EXTRA_LIBS=-L/usr/ucblib -lfast -lucb
EXTRA_LDFLAGS=-dn
used src.alt/espresso.h.sni
Fujitsu Flags:
OPT=-O3 -Ka4,dalign,lib,TMS
EXTRA_LDFLAGS=-dn
EXTRA_LIBS=-lbsdmalloc -L/usr/ucblib -lucb
used src.alt/espresso.h.apx
GNU Flags:
OPT=-O2 -w
EXTRA_LDFLAGS=-Wl,-dn
EXTRA_LIBS=-lbsdmalloc -L/usr/ucblib -lucb -lelf
used src.alt/espresso.h.apx
CenterLine Flags:
OPT=-O5 -Hit=100 -Hoff=Strength_reduce -Hpcc -w
EXTRA_LDFLAGS=-dn
EXTRA_LIBS=-lbsdmalloc -L/usr/ucblib -lucb
022.li:
Times: Apogee: 100.0 SunPro: 132.1 Advantage: Apogee 32%
Fujitsu: 137.1 Advantage: Apogee 37%
GNU: 169.0 Advantage: Apogee 69%
clcc: 139.1 Advantage: Apogee 39%
Apogee Flags:
Profile-directed dynamic feedback is used to build the timed version.
First a profiled version of the benchmark is built with:
OPT=-O -w -p -dn
The benchmark is then run with input.short/li-input.lsp, to produce
profiling data. This is processed with the prof(1) command with:
prof -cg xlisp > PROF_OUTPUT 2> /dev/null
All C files are then recompiled jointly (which permits inlining across
source files) using the profiling data with:
OPT=-O4 -w -XT=ss10/30 \
-Xinllev=3,inline=%profile50,pstat=PROF_OUTPUT,join
EXTRA_LDFLAGS=-dn
SunPro Flags:
OPT=-fast -O4 -dn -cg92 -Qoption cg -QrmW -Qoption cg -QrmX \
-Qoption cg -T5 -Qoption cg -QrmYxleval -Qoption cg -QrmYxlevarg \
-Qoption cg -QrmYevform -Qoption iropt -Mt500
Fujitsu Flags:
OPT=-O4 -Ka4,dalign,eopt,lib,TMS -x -
EXTRA_LDFLAGS=-dn
GNU Flags:
OPT=-O2 -w -finline-functions
EXTRA_LDFLAGS=-Wl,-dn
CenterLine Flags:
OPT=-O3 -Hit=200 -Hic=25 -Hpcc -w
EXTRA_LDFLAGS=-dn
023.eqntott
Times: Apogee: 13.8 SunPro: 15.5 Advantage: Apogee 12%
Fujitsu: 20.3 Advantage: Apogee 47%
GNU: 27.0 Advantage: Apogee 96%
clcc: 29.8 Advantage: Apogee 116%
Apogee Flags:
putpla.c compiled with:
OPT=-O -DSYSV -w -K -Xxopt5 -Xkap -XKstdio
all other files compiled with
OPT=-O -DSYSV -w -K -Xxopt5
Everything linked with:
EXTRA_LDFLAGS=-dn -Xkap -XKstdio
used src.alt/preprocess.c.sol2
SunPro Flags:
OPT=-fast -O4 -dn -cg92 -Qoption cg -QrmW -Qoption cg -QrmX \
-Qoption cg -QrmR -DSYSV
EXTRA_LIBS=-lfast
used src.alt/preprocess.c.sol2
Fujitsu Flags:
OPT=-O4 -Ka4,dalign,eopt,lib,TMS -x - -DSYSV
EXTRA_LDFLAGS=-dn
used src.alt/preprocess.c.sol2
GNU Flags:
OPT=-O2 -w -DSYSV -funroll-all-loops
EXTRA_LDFLAGS=-Wl,-dn
EXTRA_LIBS=-lbsdmalloc -L/usr/ucblib -lucb
used src.alt/preprocess.c.sol2
CenterLine Flags:
OPT=-O7 -Hit=100 -Hoff=behaved -DSYSV -Hoff=Globals_volatile \
-Hoff=Strength_reduce
EXTRA_LDFLAGS=-dn
used src.alt/preprocess.c.sol2
026.compress
Times: Apogee: 99.9 SunPro: 101.3 Advantage: Apogee 1%
Fujitsu: 105.3 Advantage: Apogee 5%
GNU: 101.3 Advantage: Apogee 1%
clcc: 102.0 Advantage: Apogee 2%
Apogee Flags:
OPT=-O -K -w -Xkap -XKstdio -XT=ss10/30
EXTRA_LDFLAGS=-dn
SunPro Flags:
OPT=-fast -O4 -dn -cg92 -Qoption cg -QrmW -Qoption cg -QrmX \
-Qoption cg -QrmR
EXTRA_LIBS=-lfast
Fujitsu Flags:
OPT=-O3 -w -Ka4,dalign,eopt,lib,TMS -x -
EXTRA_LDFLAGS=-dn
GNU Flags:
OPT=-O2 -w
EXTRA_LDFLAGS=-Wl,-dn
CenterLine Flags:
OPT=-O7 -Hit=200 -Hoff=Globals_volatile -Hoff=behaved
EXTRA_LDFLAGS=-dn
072.sc
Times: Apogee: 67.0 SunPro: 68.9 Advantage: Apogee 3%
Fujitsu: 69.8 Advantage: Apogee 4%
GNU: 67.1 Advantage: Apogee 0%
clcc: 69.6 Advantage: Apogee 4%
Apogee Flags:
OPT=-O -K -w -DSIMPLE -DSYSV3
EXTRA_LDFLAGS=-dn
EXTRA_LIBS=-lform -lcurses
SunPro Flags:
OPT=-fast -O4 -dn -cg92 -Qoption cg -QrmW -Qoption cg -QrmX \
-cg89 -Qoption cg -QrmR -DSIMPLE -DSYSV3
EXTRA_LIBS=-lform -lm -lcurses -lc
Fujitsu Flags:
OPT=-O4 -Ka4,dalign,eopt,lib,TMS -w -DSIMPLE -DSYSV3
EXTRA_LDFLAGS=-dn
EXTRA_LIBS=-lform -lcurses -lm
GNU Flags:
OPT=-O2 -w -traditional -DSIMPLE -DSYSV3
EXTRA_LDFLAGS=-Wl,-dn
EXTRA_LIBS=-lform -lcurses -lm
CenterLine Flags:
OPT=-O3 -w -Hit=100 -DSIMPLE -DSYSV3
EXTRA_LDFLAGS=-dn
EXTRA_LIBS=-lform -lcurses -lm
085.gcc
Times: Apogee: 133.9 SunPro: 135.4 Advantage: Apogee 1%
Fujitsu: 166.9 Advantage: Apogee 24%
GNU: 129.6 Advantage: GNU 3%
clcc: 162.6 Advantage: Apogee 21%
Apogee Flags:
OPT=-O -Xc=tolerant -w
EXTRA_LDFLAGS=-dn
ALLOCA=
used src.alt/stddef.h
SunPro Flags:
OPT=-fast -O4 -dn -cg92 -Qoption cg -QrmW -Qoption cg -QrmX \
-Xa -Dalloca=__builtin_alloca
used src.alt/stddef.h
Fujitsu Flags:
OPT=-O4 -w -Ka4,dalign,eopt,lib,TMS
EXTRA_LDFLAGS=-dn
used src.alt/stddef.h
GNU Flags:
OPT=-O2 -w
EXTRA_LDFLAGS=-Wl,-dn
used src.alt/stddef.h
CenterLine Flags:
OPT=-O3 -w -Hit=100
EXTRA_LDFLAGS=-dn
used src.alt/stddef.h
====================================
Geometric Mean for CINT92 (SPECint92)
====================================
SPECint92: Apogee: 50.4 SunPro: 46.7 Advantage: Apogee 8%
Fujitsu: 41.9 Advantage: Apogee 20%
GNU: 41.4 Advantage: GNU 22%
clcc: 40.0 Advantage: Apogee 26%
=============================================================================
=====================================================
| SPEC 1992 Floating Point CPU Benchmark (SPECfp92) |
=====================================================
For all times shown, lower numbers are better. The time is the wall-clock
runtime of the benchmark.
013.spice2g6
Times: Apogee: 685.0 SunPro: 771.7 Advantage: Apogee 13%
Fujitsu: 831.9 Advantage: Apogee 21%
Apogee Flags:
OPT=-O4 -w -Xkap -XKinline=indxx -Xsave -XT=ss10/30
EXTRA_CFLAGS=-DUNDERBAR -DSYSV
EXTRA_LDFLAGS=-dn
SunPro Flags:
OPT=-fast -O4 -cg89 -dn -Qoption cg -QrmR -Qoption cg -QrmW \
-Qoption cg -QrmX -Qoption iropt -algebra,reassoc
EXTRA_CFLAGS=-DUNDERBAR=1 -DHAS_BCOPY=1 -dalign -fsingle
EXTRA_LIBS=-L/usr/ucblib -lucb
Fujitsu Flags:
OPT=-X7 -fs -O3 -Kdalign,eval,nounroll,TMS
EXTRA_CFLAGS=-DUNDERBAR -DSYSV
015.doduc
Times: Apogee: 34.3 SunPro: 46.1 Advantage: Apogee 34%
Fujitsu: 49.5 Advantage: Apogee 44%
Apogee Flags:
OPT=-O -w -Xkap -XKinline=si -XKinff=si.f
EXTRA_LDFLAGS=-dn
SunPro Flags:
OPT=-fast -O4 -cg92 -dn -Qoption cg -T5 -Qoption cg -QrmR \
-Qoption cg -QrmW -Qoption cg -QrmX
Fujitsu Flags:
OPT=-X7 -fs -O2 -A A -Kdalign,eval,nounroll,TMS
034.mdljdp2
Times: Apogee: 110.3 SunPro: 126.3 Advantage: Apogee 14%
Fujitsu: 137.0 Advantage: Apogee 24%
Apogee Flags:
OPT=-O -w -Xfcm2,mopt4
EXTRA_LDFLAGS=-dn
SunPro Flags:
OPT=-fast -O4 -cg92 -dn -Qoption cg -QrmR -Qoption cg -QrmW \
-Qoption cg -QrmX
Fujitsu Flags:
OPT=-X7 -fs -O3 -A A -Kdalign,TMS -x -
039.wave5
Times: Apogee: 101.5 SunPro: 105.5 Advantage: Apogee 4%
Fujitsu: 116.0 Advantage: Apogee 14%
Apogee Flags:
OPT=-O -w -Xkap -XT=ss10/30
EXTRA_LDFLAGS=-dn
SunPro Flags:
OPT=-fast -O4 -cg92 -depend -dn -Qoption cg -QrmR -Qoption cg -QrmW \
-Qoption cg -QrmX -Qoption cg -Qpipe-unroll:6
Fujitsu Flags:
OPT=-X7 -fs -O2 -A A -Kdalign,eval,TMS
047.tomcatv
Times: Apogee: 40.0 SunPro: 47.2 Advantage: Apogee 18%
Fujitsu: 66.9 Advantage: Apogee 67%
Apogee Flags:
OPT=-O -w -Xkap
EXTRA_LDFLAGS=-dn
SunPro Flags:
OPT=-fast -O4 -cg92 -dn -Qoption cg -T5 -Qoption cg -QrmR \
-Qoption cg -QrmW -Qoption cg -QrmX -Qoption cg -Qpipe-group=1
Fujitsu Flags:
OPT=-X7 -fs -O3 -A A -Kdalign,eval,preex,TMS
048.ora
Times: Apogee: 64.3 SunPro: 68.0 Advantage: Apogee 6%
Fujitsu: 66.8 Advantage: Apogee 4%
Apogee Flags:
OPT=-O -w -Xfcm2,kap -XKinlc
EXTRA_LDFLAGS=-dn
SunPro Flags:
OPT=-fast -O4 -cg92 -dn -Qoption cg -QrmR -Qoption cg -QrmW \
-Qoption cg -QrmX
Fujitsu Flags:
OPT=-X7 -fs -O1 -A A -Kdalign,TMS
052.alvinn
Times: Apogee: 64.9 SunPro: 69.2 Advantage: Apogee 7%
Fujitsu: 87.5 Advantage: Apogee 35%
Apogee Flags:
OPT=-O -w -Xkap -XKarl=4
EXTRA_LDFLAGS=-dn
SunPro Flags:
OPT=-fast -O4 -cg92 -dn -Qoption cg -QrmR -Qoption cg -QrmW \
-Qoption cg -QrmX -Xa -I/l4/SC3/SUNWspro/SC3.0/include/cc
Fujitsu Flags:
OPT=-O4 -Ka4,dalign,lib,eopt,popt,GREG,TMS -x -
056.ear
Times: Apogee: 349.9 SunPro: 349.0 Advantage: SunPro 0%
Fujitsu: 386.9 Advantage: Apogee 11%
Apogee Flags:
OPT=-O -w -Xkap -XKinll=4 -XKarl=2
EXTRA_LDFLAGS=-dn
SunPro Flags:
OPT=-fast -O4 -cg92 -dn -Qoption cg -QrmR -Qoption cg -QrmW \
-Qoption cg -QrmX -Dfloat=double -Xa
Fujitsu Flags:
OPT=-O4 -Ka4,dalign,lib,eopt,popt,GREG,TMS -x -
077.mdljsp2
Times: Apogee: 105.2 SunPro: 116.9 Advantage: Apogee 11%
Fujitsu: 118.6 Advantage: Apogee 13%
Apogee Flags:
OPT=-O -w -Xfcm2,mopt4
EXTRA_LDFLAGS=-dn
SunPro Flags:
OPT=-fast -O4 -cg92 -dn -Qoption cg -QrmR -Qoption cg -QrmW \
-Qoption cg -QrmX
Fujitsu Flags:
OPT=-X7 -fs -O2 -A A -Kdalign,eval,TMS
078.swm256
Times: Apogee: 295.4 SunPro: 286.5 Advantage: SunPro 3%
Fujitsu: 336.1 Advantage: Apogee 14%
Apogee Flags:
OPT=-O -w -Xkap -XKag=a
EXTRA_LDFLAGS=-dn
SunPro Flags:
OPT=-fast -O4 -dn -Qoption cg -QrmR -Qoption cg -QrmX -depend \
-Qoption iropt -O4,fusion -Qoption cg -T6
Fujitsu Flags:
OPT=-X7 -fs -O2 -A A -Kdalign,eval,preex,TMS
089.su2cor
Times: Apogee: 285.5 SunPro: 338.3 Advantage: Apogee 18%
Fujitsu: 406.0 Advantage: Apogee 42%
Apogee Flags:
OPT=-O -XT=ss10/30 -Xalnref0,kap -w -XKinl -XKinll=1 -XKind=4 -XKag=a
EXTRA_LDFLAGS=-dn
SunPro Flags:
OPT=-fast -O4 -cg92 -dn -depend -Qoption cg -QrmR -Qoption cg -QrmW \
-Qoption cg -QrmX -Nl200
Fujitsu Flags:
OPT=-X7 -fs -O3 -A A -Kdalign,eval,preex,TMS
090.hydro2d
Times: Apogee: 300.9 SunPro: 293.2 Advantage: SunPro 3%
Fujitsu: 329.3 Advantage: Apogee 9%
Apogee Flags:
OPT=-O -w -Xkap -XKinlc -XKr=2
EXTRA_LDFLAGS=-dn
SunPro Flags:
OPT=-fast -O4 -cg92 -dn -Qoption cg -QrmR -Qoption cg -QrmW \
-Qoption cg -QrmX -Qoption iropt -par
Fujitsu Flags:
OPT=-X7 -fs -O2 -A A -Kdalign,eval,TMS
093.nasa7
Times: Apogee: 317.7 SunPro: 492.5 Advantage: Apogee 55%
Fujitsu: 520.2 Advantage: Apogee 64%
Apogee Flags:
mxm.f compiled with:
OPT=-O -w -dalign
other files compiled with:
OPT=-O -w -dalign
EXTRA_CFLAGS=-DUNDERBAR -DTIMES
EXTRA_FFLAGS=-Xkap -XKag=a
Everything linked with:
EXTRA_LDFLAGS=-dn
SunPro Flags:
OPT=-fast -O4 -cg92 -dn -Qoption cg -QrmR -Qoption cg -QrmW \
-Qoption cg -QrmX -Qoption iropt -par
EXTRA_CFLAGS=-DUNDERBAR=1 -DTIMES
Fujitsu Flags:
OPT=-X7 -fs -O -A A -Kdalign,eval,preex,TMS -x -
EXTRA_CFLAGS=-DUNDERBAR -DTIMES
094.fpppp
Times: Apogee: 159.7 SunPro: 174.7 Advantage: Apogee 9%
Fujitsu: 323.0 Advantage: Apogee 102%
Apogee Flags:
OPT=-O -w -Xmopt2 -XT=ss10/30
EXTRA_LDFLAGS=-dn
SunPro Flags:
OPT=-fast -O4 -cg92 -dn -Qoption cg -T5 -Qoption cg -QrmR \
-Qoption cg -QrmW -Qoption cg -QrmX -Qoption iropt -plus_iv
Fujitsu Flags:
OPT=-X7 -fs -O0 -A A -Kdalign,preex,TMS
===================================
Geometric Mean for CFP92 (SPECfp92)
===================================
SPECfp92: Apogee: 55.3 SunPro: 49.3 Advantage: Apogee 12%
Fujitsu: 42.3 Advantage: Apogee 31%
=========================================================================
==================================================
| Harmonic Mean and Options used for |
| Lawrence Livermore National Laboratory Kernels |
==================================================
Harmonic Mean and Geometric Mean are MFLOPS. Higher numbers are better.
Harmonic Mean: Apogee: 9.34 SunPro: 7.35 Advantage: Apogee 27%
Fujitsu: 6.93 Advantage: Apogee 35%
Geometric Mean: Apogee: 14.1 SunPro: 8.5 Advantage: Apogee 66%
Fujitsu: 7.9 Advantage: Apogee 78%
Apogee Flags: -O -Xkap,T=ss10/30
SunPro Flags: -fast -O4 -cg92 -dalign -depend -Qoption cg -T6 \
-Qoption cg -QrmR -Qoption cg -QrmW -Qoption cg -QrmX \
-Qoption iropt
Fujitsu Flags: -O2 -KTMS -X7 -fs
=========================================================================